Papers with finetuning strategy
Fine-grained Image Captioning with CLIP Reward (2022.findings-naacl)
Copied to clipboard
| Challenge: | Modern image captioning models are usually trained with text similarity objectives . reference captions often describe only the most salient objects in images . |
| Approach: | They propose to use CLIP to calculate multi-modal similarity and use it as a reward function . they propose a simple finetuning strategy to improve grammar that does not require extra text annotation. |
| Outcome: | The proposed model generates more distinctive captions than the CIDEroptimized model on text-to-image retrieval and fineCapEval. |
Parameter-Efficient Finetuning for Robust Continual Multilingual Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to Continual Multilingual Learning (CML) are based on updating models using new data in stages. |
| Approach: | They propose a parameter-efficient finetuning strategy to increase the number of languages on which the model improves after an update while reducing the magnitude of loss for the remaining languages. |
| Outcome: | The proposed model improves on the languages included in the latest update while reducing the loss of performance on the remaining languages. |